Repository navigation
new: add map_threads option for ThreadPool size in map pipe - #320
Merged
Merged
Conversation
vladimir-mencl-eresearch
force-pushed
the
add-map-threads-opt
branch
from
June 8, 2026 22:13
40a4b15 to
36fdf98
Compare
vladimir-mencl-eresearch
force-pushed
the
add-map-threads-opt
branch
from
September 2, 2026 02:42
36fdf98 to
5408928
Compare
Contributor
Author
|
Rebased on current |
Allow configuring ThreadPool size used in map pipe. Passing default value None makes ThreadPool use os.process_cpu_count(), keeping existing behaviour. Setting the value to 1 provides a workaround for IdentityPython#315 (avoid parallelism). Given that most processing happens withing Python without releasing GIL, it actually does not incur any significant penalty.
vladimir-mencl-eresearch
force-pushed
the
add-map-threads-opt
branch
from
September 7, 2026 00:02
5408928 to
158c67e
Compare
Contributor
Author
|
@theseal rebased with signed commits |
Contributor
Author
|
This PR should not be needed after #337 is merged. |
Contributor
|
Do you still agree with your earlier statement that this PR is not longer needed since change to pyuppsala? |
Contributor
Author
No longer needed as workaround for the race condition reported in #315 . However, it might still be useful if a deployment wanted to tweak the parallelism instead of going with Python default of one worker thread per discovered CPU core. Up to you whether you see it useful for the project. |
Contributor
|
Don't see any harm letting user configured their pyff in different ways. Thanks! 🙂 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Allow configuring ThreadPool size used in map pipe.
Passing default value None makes ThreadPool use os.process_cpu_count(), keeping existing behaviour.
Setting the value to 1 provides a workaround for #315 (avoid parallelism).
Given that most processing happens withing Python without releasing GIL, it actually does not incur any significant penalty.
All Submissions: